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Abstract. A Markovian model of the evolution of intermittent connections of various 
classes in a communication network is established and investigated. Any connection 
evolves in a way which depends only on its class and the state of the network, in particular 
as to the route it uses among a subset of the network nodes. It can be either active (ON) 
when it is transmitting data along its route, or idle (OFF). The congestion of a given 
node is defined as a functional of the transmission rates of all ON connections going 
through it, and causes losses and delays to these connections. In order to control this, 
the ON connections sclf-adaptively vary their transmission rate in TCP-like fashion. The 
connections interact through this feedback loop. A Markovian model is provided by the 
states (OFF, or ON with some transmission rate) of the connections. The number of 
connections in each class being potentially huge, a mean-field limit result is proved with 
an appropriate scaling so as to reduce the dimensionality. In the limit, the evolution 
of the states of the connections can be represented by a non-linear system of stochastic 
differential equations, of dimension the number of classes. Additionally, it is shown that 
the corresponding stationary distribution can be expressed by the solution of a fixed- 
point equation of finite dimension. 

1. Introduction 

The Internet network can be described as a very large distributed system, which manages 
the dynamic exchange of data transmission through connections established between nodes 
of the network under a very dynamic, random, scheme. Nodes cannot cope at all times 
with the huge amount of data transmitted through them by the varying connections, and 
congestion events occur, causing losses and delays to the connections. 

AIMD algorithms (Additive Increase, Multiplicative Decrease) in the TCP protocol reg- 
ulate Internet traffic: a connection increases gradually its throughput as long as congestion 
does not occur along its route (Additive Increase), but decreases it brutally when such an 
event takes place, usually by cutting it in half (Multiplicative Decrease). 

An important practical issue is to obtain a better qualitative and quantitative under- 
standing of such algorithms, and devising and analyzing pertinent mathematical models 
may be very helpful. There are many serious challenges, see Graham and Robert [7 for a 
discussion of the literature in this domain. Some objectives are to: 

(1) Propose a stochastic model for the arrivals and durations of connections. 

(2) Propose a stochastic model of flow control by AIMD algorithms; see, e.g., Dumas 
et al. [5] and Guillemin et al. [S]. 

(3) Describe the state of the network, including the large number of connections inter- 
acting at its nodes through the congestion they create, in a mathematically tractable 
way. 
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Let us be more specific. A client at a node initiates a connection to a server at another 
node, in order to transmit data for some specific purpose. Each data packet is routed from 
source to destination through intermediate nodes, using local routing decisions depending on 
the instantaneous state of the network. The connection adapts to the congestion encountered 
by its packets by adjusting its throughput as above, and ends once all the data has been 
completely transferred. 

The same client may well repeat these steps at various instances with the same server, 
purpose, and reaction characteristics to network conditions, and thus intermittently establish 
connections of the same kind. For simplicity of expression, we call "user" such a client-server- 
purpose-rcaction combination. 

Graham and Robert [7] introduced and analyzed a Markovian network model, further 
studied in Graham et al. [8], constituted of J nodes and hosting K classes of permanent 
connections. This paper extends this to K classes of users establishing intermittent con- 
nections in an ON-OFF pattern. An OFF user is idle, and switches ON after a random 
duration. While ON, the user transmits under an AIMD scheme (after establishing a con- 
nection) and switches OFF (cutting the connection) after a random duration depending on 
throughput (related to node load). In particular, the number of on-going connections varies 
with time. 

This gives a possible answer to Point 1 above; ON-OFF users are quite natural in models 
for data transmission in the Internet, for instance a web transaction (http connection) can 
be thought of as a succession of file transfers between server and client. For Point 2, the 
load of a node may be dehned as a linear functional of the throughputs of all connections 
going through it, and the loss rate as a function of these quantities, as in [7]. 

Point 3 is achieved by proving a mean-field limit result, propagation of chaos, which we 
describe roughly. When the numbers of users in each class converge to infinity with limit 
ratios, for adequate initial conditions, asymptotically users evolve independently, and the 
limit behavior of users of class k for 1 < k < K is described by a nonlinear (McKean- 
Vlasov) stochastic differential equation (SDE) in M*£, the effect of the interactions on the 
connections being encoded in the non-linearity of the SDE, as in [7]. 

The introduction of intermittent connections significantly changes the problems investi- 
gated in [8]. A cemetery state —1 is added to encode the fact that a user can be OFF, 
and notably creates technical problems concerning the mandatory Lipschitz properties for 
mean-field convergence. Moreover, the limit stationary distributions are still characterized 
by finite vectors solving a fixed point equation, but their expression is much more com- 
plicated than before; it is related to the resolvent of a generic one-dimensional Markov 
process. 

Section [2] introduces more precisely the stochastic model of the network, and the mean- 
field scaling. Section [3] gives the main properties of the limiting nonlinear SDE. Section |4] 
establishes the mean-field result. Section [5] studies equilibrium properties of the limit. 
Acknowledgements 
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2. Markovian multi-class model for the network 

2.1. Classes of ON-OFF users, throughput, node load. Recall that "user" denotes 
a possible kind of network use by a source-destination pair, and hence corresponds to the 



MULTI-CLASS INTERMITTENT CONNECTIONS 



3 



behavior, with respect to congestion, of its packet routes, rate of increase of throughput, 
drop rate, etc. 

The stochastic network is constituted of J > 1 nodes and hosts K > 1 classes of users. 
There are N k > 1 users of class k for 1 < fc < K, which bounds the varying number of 
possible on-going transmissions of this class. The notations 

N=(N 1 ,...,N K ), \N\ = N 1 + --- + N K , 

will be used, and \N\ is the total number of users in the network. 

Users alternate between being active, or ON, and inactive, or OFF. An ON user has data 
to transmit, and establishes a connection in the network, with throughput controlled by a 
dynamically self-managed "window size" with values in R + until it is done. A fictitious 
window size of —1 will be assigned to OFF users. The state space for a user is R + U { — 1}, 
with the trace of the usual metric. The cemetery state (with resurrections) brings new 
problems with respect to Graham and Robert [7J, but the choice of —1 simplifies notations. 

The way ON users utilise the nodes depends only on their classes, and is given in terms 
of an allocation matrix 

A = (A jk , 1 < j < J, l<k<K), Ajk € R+ , 

as follows: at node j, the instantaneous weighted throughput of a class k user in state 
to G R+ U { — 1} is given by 

AjkW + := A jk wl{ w > } , 
which vanishes for OFF users and is proportional to state (window size) for ON users. 

Hence the load at node j is given by the total weighted throughput of all users, and when 
the n-th of class k is in state w n ,ki by 

K N k 

(i) «j = EE^<n i<s<j- 

k=l n=l 

The node load vector 

U = (Uj, 1 < j < J) 

is an important descriptor of the congestion of the network, and is indirectly sensed by the 
users through their losses. It is a linear function of the states of the ON users. 
In a simple example, mapping classes to routes, 

Ajk = 1 if node j is used by a class k user, and Ajk = otherwise. 

More general matrices allow to model, e.g., different utilisations of a given node by different 
classes, or packets of a given user taking varied routes in the network. 

2.2. Connection initiation and termination, congestion control. Transitions be- 
tween ON and OFF states are given by functions : R+ — > R+ and n k : M+ x — > R + and 
laws a.k on R + , for 1 < k < K. When the node load vector is in state u — (uj, 1 < j < J), 
see ([T]), then in class k: 

• any OFF user (in state —1) is turned ON at instantaneous rate Afc(it) with new state 
w chosen according to a k (dw) , and establishes a connection for data transmission, 

• any ON user in state w > is turned OFF (to state —1) at rate /j,k(w,u), and 
terminates the connection. 

The throughput of connections are determined by the instantaneous states of the users 
that have initiated them; in a simple example, their initial laws a k are delta masses, such as 
5q. They cause congestion at the nodes of the network, see ([1]), which is controlled by ON 
users by regulating their state (throughput). This involves functions a k ■ R+ x R^ — > R + 
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and bk : R+ x M.+ — > R+ and constants rk S [0, 1], for 1 < fe < K; when the node load vector 
is u = (v,j, 1 < j < J) and the state of a class fc user is w > 0, then instantaneously: 

• the state increases continuously at speed ak{w,u), 

• the state jumps from w to r^w at rate bk(w,u), corresponding to the occurrence of 
a loss in the transmission due to congestion (usually ffe — 1/2). 

The study in Graham and Robert [7] considered permanent connections, corresponding 
to having /j,k(w,u) = and all users ON at time 0. Then, users can be identified with 
connections and have state space R+, and there is no use for Xk and ctk- 

Natural special forms for the parameters. A simple case is when the throughput rate is 
proportional to the state (window size) of any user, so that there are functions Vk and j3k 
from to R+ such that 

(2) fik(w,u) = wvk{u) , b k (w, u) = w(3 k (u) . 

A special case has constant functions Xk (u) = Xk > and Vk (u) = Vk > (with slight abuse 
of notation), so that any class k user goes ON at rate Afe, is required to transmit a quantity 
of data which follows an exponential law with parameter Vk at instantaneous throughput 
rate given by its window, and then goes OFF. Also, a natural case has 

(3) a k {w,u) = a k (u) 

regardless of w (with slight abuse of notation), and related to the inverse of the round-trip 
time (RTT) in the network when its load is given by the vector u. 
A natural sub-case of the above is when 



where, for class k users, Tfe > is the RTT between source and destination and 5k > is 
the loss rate in a non-congested network, and tjk(uj) > is the supplementary RTT delay 
and djk(uj) > loss rate at node j when its load is Uj. We may expect tjk{uj) and djk{uj) 
to behave linearly in Uj, at least for large Uj. 

Hence, care will be taken to include cases where bk(w,u) may have a quadratic behavior 
in the assumptions to be made. 

General forms of the parameters Xk, Hk, a-k, bk, and are used to model, e.g., balking 
(at arrival and later) in a congested network, more complex relations between window size 
and throughput and losses, sequences of losses due to congestion, slow start, RED, etc. For 
instance, a last very simple case is Xk(u) = Afe > and Hk(w,u) = [1% > 0, corresponding 
to someone popping in and out to browse the Internet in his spare time. 

2.3. Markov process and its SDE representation, mean-field scaling. A Markov 
process describing this model is given by 




W N (t) = (W£ k (t), 1 < n < N k , 1 < k < K) , 



t>0, 



where 



W„ W fe(t)eR+u{-i} 
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is the state of the n-th user of class k at time t. We choose to represent it as the solution 
of the Ito-Skorohod stochastic differential equation (SDE) 



(5) 



d W ntk (t) = 1 {w n (t_)=_i} / (1 + w)l {0<z<Xk{U N {t _ m An tk {dw,dz,dt) 



+ 1 { w/ " fc (*-)>°} 



o fc (WA(t-)»^(*-))* 



N i 



(1 - r*)W£*(t-) / 1 [o<z<b k (w N k (t-),u N (t-))} N n> k{dz, dt) 



~ (1 + W n>k {t-)) I J{ < z < Mfc (W~ fe (t-) ,u»(t-))} 1> n ,k{dz,dt) 
l<k<K, 1 <n< N k ; 



K 



U N (t) = (U? (t), 1 < j < J) , Uf(t) = A jk J2 W nAt) + > 



fc=l i=l 



driven by Poisson point processes A n ,k on with intensity measure a k (dw) dz dt and M nk 
and P ni fe on with intensity measure dzdt, all independent. See Graham and Robert [7] 
for a discussion of the related simpler model for permanent connections. The following can 
be proved with standard arguments. 



Proposition 2.1. If the functions a k are Lipschitz and the functions b k , and fj, k are 
locally bounded, 1 < k < K , then there is pathwise existence and uniqueness of solution 
for the stochastic differential equation (J5J, and the corresponding Markov process is well 
defined. 



2.4. The mean-field asymptotic regime. The high-dimensional system of coupled sto- 
chastic differential equations ([5]) does not seem to be mathematically tractable as such, and 
an asymptotic study, reducing its dimension \N\ to the number of classes K, is used to 
investigate the qualitative properties of solutions. 
We consider the mean-field scaling in which 



(6) 



N k N k 
Nk ^°°> W\'- = N 1 + --- + N K 



Pk ■ 



1 < k < K, 



(necessarily (p k )\<k< 

are scaled by a factor \N\, i.e., U N is replaced by U 



x is a probability vector) , and moreover the capacities of the resources 

hrU N in (J3J) so as to have a 



■N 
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non-trivial limit. This procedure leads to the reseated SDE 



(7) 



dwZ k {t) = i Wfc(t -)=-i } / + 



+ ^{w^t-)^)} 



L {0< Z <A fc ((7 (i-))} 



OfcfW&tt-),^-))* 



-(l-r fe )W£ fe (t-) 

-(i + w^CH) 

1 < fc < AT , 1 <n< N k ; 



1 {o<z<b k (w£ h (t-),u N (t-))} J ^ n > k ( dz > dt ") 



L {0<z<» h (W«(t-),U"(t-))} 



V n ^ k (dz,dt) 



U N (t) = (Uf(t),l<j<J), 



^(*) = ^(*)=E^<(')- 



This is a multi-class mean-field system, in interaction through the vector 

W N {t) = (Wf (0,1 <k<K) 



of the empirical means of the class throughputs, via the vector U (t) of the scaled loads. 
For 1 < k < K , it is natural to introduce the empirical measure for the processes associated 
to class k users, given by 

1 N k 



A? 



where 5 



(x(t),t>0) 



denotes the Dirac mass at the function i G M + a;(t) G R+ U {— 1}, and 



7 JV, 



in particular W k (t) — (w + ,A k v (t)(dw)), where A k (t) is the marginal of A^ at time t 



2.5. Notations and conventions. For x 

for some M G N and T > 0, let 



\x\\ = max ar m , 

Km<M 



(x TO , 1 < m < M) G 



||u||t= sup \\v(s)\\ 

0<s<T 



»M 



and i) 



Depending on the context, the function v = (v(t), t > 0) may be denoted 

(u m (t), 1 < m < M,t > 0) or (v m (t), 1 < m < M) or (v(t)) . 



If H is a complete metric space, 2?(R+, 7J) denotes the Skorohod space of functions with 
values in H, right-continuous with left limits at any point of R+, see Billingsley [3]. 

The r.v. Af has values in the set V(V(R + ,R + U {-1})) of probability measures on 
2?(R + ,M + U {-!}). 



3. Analysis of the nonlinear limit process 

3.1. Heuristic derivation of the mean-field limit. For (jJJ), the symmetry properties 
within each user class lead us to expect a mean-field convergence phenomenon as N gets 
large, for appropriately converging initial conditions: 

• the processes W^ k for 1 < k < K and 1 < n < N k should become independent and 
converge in law to processes Wk (depending only on the class k), 
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• and the empirical measure should converge in law to a deterministic limit given 
by the law of the same process Wk, 

where the stochastic process 

{W{t),t > 0) = ((W k {t),t > 0),1 < k < K) 
is the solution of the nonlinear, or McKean-Vlasov, Ito-Skorohod SDE 



(8) 



dWk(t) = l{w fe (t-)=-i} / 0- + w)l{o<z<\ k (u w (t))}^k{dw,dz,dt) 



+ ^{W k (t-)>0} 



a k (W k {t-),u w (t)) dt 



(l-r fe )Wfc(t-) / l { o <z<bk{Wki t-),u w (t))}Nk(dz,dt) 



(1 + W fc (t-)) / l {0<z<tlk{Wk{t _ huw(tm V k (dz,dt) 



l<k<K; 

u w (t) = (u w ,j(t), l<j<J), 



K 



,{t) = Y J A 3 kPknwk{t) + )i 



fc=i 



with intensity measure ak(dw) dz dt and A4 



driven by Poisson point processes Ak on 
and T>k on with intensity measure dzdt, all independent. 

The interaction between coordinates depends on the mean throughput vector 

(u w (t),t > 0) 

which is a linear functional of the mean class output vector 

E(W{t)+) = E(W k {t)+, 1 < k < K) = (w + ,C(W(t))) . 

In particular, the infinitesimal generator of the process (W(t),t > 0) depends, at time t, on 
the law of W(t) and not only on the value taken by the sample path, as it is usually the 
case. Using this generator, there is a nonlinear martingale problem formulation for the weak 
interpretation of Equation ([8]). 

Some properties of the solutions of the SDE ([8]) are analyzed in this section. The mean- 
field convergence results will be the topic of Section 



3.2. Existence and uniqueness results. The following proposition is the central tech- 
nical result used to establish the existence and uniqueness of solutions of Equation ([5]), as 
well as the mean-field convergence result. For this last purpose, it is convenient (but not 
essential) to obtain a bound C(t) which does not depend on 

Proposition 3.1. For 1 < k < K, let the junctions be bounded and au, b^, and 

be Lipschitz, and the laws a k have a first moment m k — f w ctk(dw) < oo. and let A k 
be Poisson point processes on M.^_ with intensity measure ak(dw) dz dt and Mk and T>k be 
Poisson point processes on with intensity measure dz dt, all forming an independent 
family. For any u = (u(t),t > 0) in C(IR-|-,R• 7 ) and (M + U { — \}) K -valued random variable 
Xq independent of the Poisson point processes, let 



4(X ,u) = (X(t),t> 0) = ((X k (t),t> 0),1 < k < K) 



cS 



CARL GRAHAM AND PHILIPPE ROBERT 



be the solution starting at X(0) = Xq of the stochastic differential equation 
dX k {t) = l {Xk (t-)=-i} / (l+w)t {0<z<Xk ( u ( tm Ak(dw,dz,dt) 



+ l{x fe (t-)>o} a k {X k (t-),u(t)) dt 

- (1 -r k )X k {t-) J l{ Q<z<bk (x k (t-)Mt))}Nk{dz,dt) 
-(l + X k (t-)) / l {0<z<t , k ^ Xk (t~),u(t))}' D k(dz,dt) 



1 < k < K . 

Let u — (u(t),t > 0) and v! — (u'(t),t > 0) be in C(M. + ,M. J ), and initial values X and 
Xq be independent of the Poisson point processes. Then, for all t > 0, 



(9) E \\<f>{X ,u)- <f>(X^,u') \\ t X ,X 



< 



\\X G -X' \\ + / C(s)\\u-u'\\ s ds 



where C(t) — A(l + 1 + \\u\\t + \\Xq\\ + ||A"q||) f or some constant A < oo depending only on 
a-k, b k , \ k , fi k , and m k , 1 < k < K. 



Proof. Any solution (Y(t),t > 0) = (Y k (t), 1 < k < K,t > 0) of §£J, for any u and initial 
condition, satisfies the a priori affine growth bound 

(10) E"[Y k (t)+] < max(y fe (0)+,TO,.) + \\a k \\t, 

which notably allows to prove, classically under the present assumptions, existence and 
uniqueness of solutions for Q£ M D recursively from jump instant to next jump instant, since it 
implies that these cannot accumulate. Let 

X(t) = 0(X o ,u)(t), X'(t) = 0(X£, «')(*), t>0, 

and E° denote the conditional expectation given Xq and Xq. Then 

(11) \\X k - X' k \\ t < \X k (0) - X' k (0)\ + h(t) + I 2 {t) + h(t) + h{t) 



for 



h{t) 

hit) 
hit) 

hit) 



(1 + w)|l{X fc ( S -) = -l}l{0<z<A fc (u(s))} 

- l{x£(s-)=-i}l{o<2<A fe («'(s))}| A k (dw,dz,ds) , 
{x k (s-)>o}a k iX k is),uis)) - l {X ' k ( s -)>o}a k iX' k is),u' is))\ ds , 

X k is-) + l {0<z<bk{Xk (s-)Ms))} 

- x 'k( s -) +1 {o<z<b k (x k (s-),u'(s))}\Nkidz,ds) , 

r 

(1 + X k {s~))t {0<z<tlk(Xk{s _ )Ms))} 

- (1 + x 'k( s -)) 1 {o<z<n k (x k (8-),u'(s))}\ V k {dz,ds) 



where the fact that l{ x > yx = x + and l{ x >o} (1 + x) = 1 + x for x in R+ U { — 1} is used for 
regularizing some integrands. 
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Compensating the Poisson point process A k and the inequality 



(12) 



\-{x = -l] ~ l{as'=-l}| = |l{x>0} - l{a;'>0}| < \x - x'\ , X, x' £ R + U {-1} 



(the functions 1r + and l{-i} are 1-Lipschitz on R + U { — 1}) yield 



E°[/i(i)] = E C 



(1 + w)|l{X fc ( s ) = -l}l{0<z<A fc («(s))} 



< (1 + m k )E° 



L{X£( s )=-i}l{0<*<A h (u'(*))}| a k (dw) dz ds 



|l{X fc (s)=-l} - l{X;( s )=-l}|l{0< Z <A fc («(s))} 



+ 1 {^(s) = -l}| 1 {0< Z <A fc («( ;i ))} - l{0<^<A fe (u'(6))} 



dz ds 



< (1 + m fe )E° 



|jffc(s) -X£(s)|A fc (u(*)) + |A fc (u(«)) - A fc (u'(s))| 



The singularities due to the cemetery state —1 now cause further difficulties. A key step 
in the proof is that, since the functions a k , b k , and fi k , defined on R + x , can be extended 
arbitrarily to (R+ U { — 1}) x without altering ^2, -^3, and I4, we may introduce adequate 
Lipschitz extensions for such functions / by setting /(— l,u) = /(0,w), i.e., 

f(x, u) = f(x+,u) , (x, u) e (R+ U {-1}) x R J + . 

Since x 1— > x + is 1-Lipschitz, the extensions are Lipschitz on (R + U { — 1}) x R+ with same 
Lipschitz coefficients as the original function on R + x WtL. (Note that the extension vanishing 
on { — 1} x M. J may well not be Lipschitz on (R + U { — 1}) x Ri, see for instance (x,u) £ 
R+ x R J x + u.) 

In the sequel, we use these adequate Lipschitz extensions of a k , bk, and /ifc, which play 
an important role in regularizing the integrands, and invisibly simplify greatly the compu- 
tations. Simple computations and (TT21 yield 



E°[/ 2 (i)] =E° 



|l{x fc ( s )>o}afe(A fc (s),u(s)) - l{ X '(s)>Q}ak(X' k (s),u'(8))\ds 



< 



< E° 



\^{x k (s)>o} - ^{x' k (s)>o}\ak(X k {s),u{s)) 
+ ^-{X' k ( s )>o}\ak(X k (s),u(s)) - a k (X' k (s),u'(s))\ 
\X k {s) - X' k (s)\a k (X k (s),u(s)) 
+ \a k (X k (s),u(s)) - a k (X k (s),u'(s))\ dzds 



dz ds 
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compensating the Poisson point process A4 yields 



E°[I 3 (i)]=E 

< E° 

< E° 



J J \Xk(s) + l{ 0<z<bk (x k ( s ),u(s))} - X' k (s) + t {0<z<bk{X ' k (s),u'(s))}\dzds 
X k (s) + - X' k (s) + \l {0<z<bk{Xk ( s ),u( s ))} 

+ X' k (s) + \l {Q<z<bk{Xk (s),u(s))} ~ 1 {0<z<b k (X' k (s),u'(s))}\ dzds 

* X k (s) - X' k ( S )\b k (X k (s),u(s)) 

+ X k (s)+\b k (X k (s),u(s))-b k (X k (s),u'(s))\ 



ds 



and similarly, replacing x + by 1 + x in the computations, 



E°[h(t)] < E° 



\X k (s)~X' k (s)\^ k (X k ( S ),u( S )) 

+ (l + X' k (s))U (X k (s) , u(s)) - n k (X' k (s) , u'(s))\ 



ds 



Going back to (HH), the bounds on E°[7i(t)], E ^)], E°[I 3 (t)], and E°[I 4 (t)), the affine 
growth bound (|10p. and the Lipschitz bounds which remain valid for the extensions of a k , 
b k and on (R + U {-1}) x M^), yield 

E°[||X fe - AT^Ht] < |X fc (0) - X£(0)| + f C k (a) (\\u(s) - u'(s)\\ + E° [\\X k X k \\ 8 ]) ds 

Jo 

where Ck(s) = A k (l + s + \\u\\ s + \\X \\ + \\Xq\\) for some constant A k < oo depending only 
on a k , bk, Xk, fJ-k, and mk, 1 < k < K . The Gronwall Lemma then yields 

ft 



E°[\\X k -X' k \\ t } < 



\X k (0)-X' k (0)\+ / C k (s)\\u(s)-u'(s)\\ds 



< 



\X k (0) - X' k (0)\ + / C k (s)\\u-u'\\ s ds 



e f*C k (s) ds 
tC k (t) 



and the proof of the proposition follows. 



□ 



One of the main motivations of this study is to obtain results which are valid for functions 
\ik and bk of the form given in ^ and Q. Such functions may have quadratic behavior, and 
Proposition ^ . 1 1 must be adapted to this case, which replaces ||^o|| + H^oll by ||Xo|| 2 + ||^oll 2 
in the bound for C(t), but for simplicity this will be stated only when needed. 

In order to control the evolution of (W(t),t > 0) and describe its stationary behavior, 
the simple assumption that initial conditions be uniformly bounded is not satisfactory. For 
these purposes, exponential and Gaussian moment assumptions are introduced. 



Condition (C) It is said to hold for a family {X®, 9 £ 0} of (R + U { — 1}) K -valued r.v., for 
the functions b k : R + x 1 j 4 1 + and [i k : R+ x R^ ->■ R + for 1 < k < K, and for e > 0, 
when at least one of the two following conditions is satisfied: 

(1) The functions bk and are Lipschitz for all 1 < k < K , and the X® for 6 G Q 
have a uniform exponential moment of order e: 



supE[exp(e||X | 



< oo . 
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(2) There are Lipschitz functions j3 k ■ R+ — > R+ and v k : R+ — > R+ such that 

bk(w,u) = w/3 k {u) , Hk(w,u) = wv k (u) , l<k<K, 

and the X® , 9 G 0, have a uniform Gaussian moment of order e: 

supE[exp(e||X 6 '|| 2 )] < oo . 
see 

The following theorem establishes the existence and uniqueness result for Equation ([5]). 

Theorem 3.2. // the functions a k ■ R+ X R^ — ► R+ are bounded and Lipschitz and \ k : 
R'{ -> R+ are Lipschitz, 1 < k < K , and if Condition (C) holds for the (R+ U {-1}) K - 
valued r.v. Wq, the functions b k and [i k for 1 < k < K , and e > 0, then there is pathwise 
existence and uniqueness of the solution (W(t),t > 0) of the nonlinear stochastic differential 
equation starting at Wg . 

In this case, the solution depends continuously on the initial condition in the following 
way: if(W(t),t > 0) and (W'(t),t > 0) are solutions o/© with respective initial conditions 
Wo and Wq having the same moment in Condition (C), then for T > there exists a 
constant At such that 



(13) E[\\W-W'\\ T } :=E 



sup \\W(s) - W'(s) 



s<T 



< A T E 



Jl[exp( E (\\W \\ e + \\WZ\\ e ))] 



where 1=1 or 2 depending on whether the moment in Condition (C) is exponential or 
Gaussian. 

Proof. The proof follows mutatis mutandis the proof of Graham- Robert [7J Theorem 4.2], 
substituting Proposition 13. II to [7j Proposition 4.1], and we give some details about this. If 
(1) of Condition (C) holds, then © (given in Proposition 13- ip is substituted to [7J (4.3)] 
(given in [71 Proposition 4.1]). If (2) of Condition (C) holds, then Proposition (|3. II) and its 
proof must be adapted as in the end of the proof of 7, Theorem 4.2]: the term E°[I 3 (i)] 
must be bounded using 

\b k (X k (s),u(s))-b k (X' k (s),u'(s))\ = \X k (s)p k (u(s)) - X' k {s)p k (u'(s))\ 

< X k (s)\p k (u(s)) - (3 k (u'(s))\ + p k (u'(s))\X k (s) - X' k (s)\ 

and similarly for E° [I4 (t)] , and the rest of the proof is analogous, the supplementary multipli- 
cations by X k (s) and X' k (s) being handled by using the Gaussian moment assumption. □ 



4. Mean-field limit theorem for converging initial data 

It is said that multi- indices N = (N k , 1 < k < K) go to infinity, denoted by N — > 00, 
when 

min Nk 00 ■ 

l<k<K 

This is the case in the mean-field scaling, where ([6]) gives the relative growth rate of the 
numbers of class k users. In this context, the notions of exchangeability and chaoticity play 
a fundamental role. These properties are classical for single-class systems, see Aldous pQ 
and Sznitman [T^] for example, but need to be extended to the present multi-class network. 
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Definition 1. The family ofr.v. (X n . k , 1 < n < N k , 1 < k < K) is multi-exchangeable if its 
law is invariant under permutation of the indexes within the classes: for any permutations 
°~k of {X, ... , N k } for 1 < k < K , there holds the equality of laws 

C(X ak(nhk , l<n<N k ,l<k<K)= £{X n . k , 1 < n < N k , 1 < k < K) . 

A family (X^ k , 1 < n < N k , 1 < k < if) of multi-class random variables indexed by N = 
{N k , 1 < k < K) 6 N A ' is Pi®- • • <8> Pk -multi- chaotic, where each P k is a probability measure, 
if for any m > 1 there holds the weak convergence of laws 

lim C(X" h , Kn<m,Kk<K)= Pf m ® • ■ • <g> P® m . 

Hence, such a family of systems is multi-chaotic if and only if it becomes asymptotically 
independent with particles of class k having law P k . A surprising result, not used in this 
paper, is that a family of multi-exchangeable multi-class systems is multi-chaotic if and only 
if the restriction to each class is chaotic, see Graham [51 Theorem 3]. 

The following theorem is the main mean-field convergence result. The underlying topol- 
ogy on the corresponding functional space is uniform convergence on compact sets. 

Theorem 4.1. It is assumed that: 

(1) the mean-field regime ([B]) holds: the multi-indices N = (Ni, . . . ,Nx) go to infinity 
so that 

lim = p k , 1 < k < K , 

(2) the M+ U {— l}-valued random variables 

(W£ k (0),l<n<N k ,l<k<K) 

are multi- exchangeable and Pi,o® - • ■ ® PK,o- m ulti-chaotic, where P k $ is a probability 
distribution on R + U { — 1} for 1 < k < K , 

(3) the functions a k : R + x R-[ -> M + are bounded and Lipschitz, the functions X k : 
M.^_ — > are Lipschitz, and Condition (C) holds for the random variables 

{Wf(Q),N 6 N K } = {(W^ k (0), 1 < k < K), N g N K } , 
the functions b k and /i k for 1 < k < K , and e > 0. 
Then, for 1 < k < K , in the sense of processes 

N k 





lim E 

iV->-|-oo 



^rYw"{t)-nw k {t)) 



N k ^ 

n=l 



and the family of processes 

((W£ k (t), t>0),l<n<N k ,l<k<K) 

given by the solutions of the SDE (JT]) starting at the initial conditions (W^(O)J is multi- 
exchangeable and Pw -multi- chaotic, where P\y — Pwi ® ' ' ' ® Pw K is the law of the solution 

(W(t),t > 0) = ((W k (t),t > 0),1 < k < K) 

of the nonlinear SDE ([8]) with initial distribution Pi.o ® ■ ■ ■ ® Pk.o- 

In particular, (W^ (t),t > 0) := {{W^ k {t),l < k < K),t > 0) converges in distribution 
to (W(t), t>0) = ((Wfe(t), 1 < k < K), t > 0) when N —> oo, alln>l. 

Proof. As in the proof for Theorem 13. 2\ the proof follows the proof of Graham-Robert [TJ 
Theorem 5.1], substituting Proposition 13.11 to Proposition 4.1] if (1) of Condition (C) 
holds, and adapting appropriately Proposition 13. II if (2) of Condition (C) holds. □ 
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The following is a simple consequence of the above result and of exchangeability. 



Corollary 4.2. Under the assumptions and notations of Theorem ^. 1\ the convergence in 
law of the empirical distributions 

limAf=P Wfc , l<k<K, 

iV— >OG 

holds for the weak topology on V , R+ U { — 1})) with X>(R + ,IRl + U { — 1}) endowed with 
the Skorohod topology. 

5. Equilibrium behavior 

5.1. Stationary distributions for the nonlinear SDE. The basic equilibrium properties 
of (W(t), t>0) = ((W k {t),t > 0), 1 < k < K), solution of the nonlinear SDE © describing 
the limit user evolution, are now succinctly studied, in particular its stationary distributions. 

Note that, due to the non-linearity, (W(t),t > 0) is not a homogeneous Markov process, 
and therefore the classical theory concerning the convergence toward equilibrium of Markov 
processes does not apply. 

The coordinates of the process (W(t), t > 0) evolve independently, and are coupled through 
the presence in the coefficients of Equation ([8]) of the load vector (uw (t),t > 0) given by 

K 

(1) uw(t) = (uw,j(t), l<j<J), u Wj (t)=Y / A 3kPk E(W k (t) + ). 

fe=i 

If a stationary distribution exists, and is taken as the initial distribution, then uw(t) is 
constant in t, and hence (W(t),t > 0) is a stationary process with independent coordinates. 

One begins with the analysis of the equilibrium of a single process (W k (t),t > 0)) when 
the load vector uyy(t) is replaced by a constant u, then examines the fixed-point equation 
resulting from the coupling of the coordinates through Relation ([I]), which in equilibrium 
does not depend on t. 

5.2. Invariant measures for a generic process. Throughout this section a load vector 
u = (uj , 1 < j < J) G is fixed without further mention, and simplified notations are 
used, dropping the index k from notations. Normal notation will be resumed in the following 
section. 

One studies the invariant measures for the classic Ito-Skorohod SDE on M + U { — 1} 



dW(t) = l {w(t _ )= _ 1} J (1 + w)l {0<z<Ku)} A(dw, dz, dt) 

+ l{W(t-)>o} a(W(t-),u) dt-(l-r)W(t-) J l{ 0<z<b (w(t-), u )} N(dz, dt) 
(2) - (1 + W{t-)) [ l {0<z<K w(t-),u)} V{dz, dt) 



driven by Poisson point processes A on with intensity measure a(dw)dzdt, and T> and 
AT on with intensity measure dzdt. An important element of the study will be the SDE 
on M + , corresponding to the evolution of a permanent connection, 

(3) dV(t) = a {V{t-),u) dt~{\- r)V(t-) J l {0<z<b(V (t-),u)} N{d Zl dt) . 

For definitions and results on Harris ergodicity, see Nummelin [10] , Asmussen 2; . Actually, 
classical positive recurrent state techniques are used. 

Theorem 5.1. Assume that 
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(1) the function x i— > a{x,u) is Lipschitz bounded, and mi{a(x,u) : x > 0} > 0, 

(2) the function x i— > b{x,u) is locally bounded, and ini{b(x,u) : a; > xo} > /or some 
xo > 0. 

Then, for given initial conditions, there exists a unique solution (W(t),t > 0) of ([2]), and a 
unique solution (V{t),t > 0) of ([3]), and this defines Markov processes. Moreover, (V(t),t > 0) 
/ias a positive recurrent state and hence a stationary distribution tt v , and is in particular 
Harris ergodic. 

Proof. The existence and uniqueness result of solution and the Markov property are classical. 
The hitting time of xq by (V(t),t > 0) is defined by 

T X0 = inf {t > : V{t) = x , 3s < t, V{s) / x } . 
Let r) := m{{b(x,u) : x > xq}, and (V(t),t > 0) be the solution of the SDE 

(4) dV(t) = a(V(t-), u) dt-(l- r)V(t-) J 1 {0<z<v} Af(dz, dt) 

(existence and uniqueness are classical). If V(0) > V(0) > xq then it is a simple matter to 
construct a coupling of (V(t),t > 0) and (V(t),t > 0) such that the relation V(t) > V(t) 
holds as long as V(t) > xq- The Markov process (V(t)) is Harris ergodic; see Dumas et 
al. [5] for the existence of a stationary distribution satisfying the coupling property, and 
Chafa'i et al. [4j for more on long-time convergence. The hitting time below xq is therefore 
integrable for (V(t)), and hence for (V(tj). 

Now if V(0) < xq then, since there are no upward jumps and x i— > a(x, u) is lower bounded 
away from and x i— > b(x,u) is bounded on the compact set [0,xo], it is a simple matter 
to show that the hitting time of xo by (V(t)) is integrable. One therefore concludes that 
E XQ (T Xo ) < cxd, and xq is positive recurrent, and classical results conclude the proof. □ 

Note that (W(t),t > 0)) (as defined in this section) is an homogeneous Markov process, 
and can be seen as a solution of the SDE © killed at some random instants and regenerated 
after an exponentially distributed duration of time with parameter X(u). Because of this 
regenerative structure, the stationary distribution of the SDE ([3]) will not come into play as 
it did in Graham and Robert [7] . 

Theorem 5.2. Let the functions a and b satisfy the assumptions of Theorem I5.il and 
moreover X(u) > and J /i(x, u)ir v (dx) > 0. Let (V(t),t > 0)) be the solution of ([3]) 
starting at V(0) with law a(dw), and 7 be the positive measure on K + U { — 1} defined by 

(5) J f(x) 7 (da:) = /(-l) + \{u) E (f(V(t)) exp (- J fi(V(s),u) ds^j \ dt 

for all non-negative B or el functions f on R + U{ — 1}. Then 7 is the unique invariant measure 
of the Markov process (W(t),t > 0)) defined by for all t>0, 

J E x (f(W(t)))j(dx) = J f(xMdx) 

for all non-negative Borel functions f on R + U { — 1}. 
Proof. Let the stopping time 

T-i = m£{t > : W(t) = -1, 3s < t, W(s) ^ -1} 
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be the cycle time of (W(t),t > 0) between two visits to —1. Classically, if P(r_i = oo) = 
then it is well known that the measure 7 defined by 

(6) J f(x) j(dx) = E_! fT'* f(W(s)) ds 

for all non- negative Borel functions / on R+ U { — 1} is the unique invariant measure for 
(W(t),t > 0), see e.g. Asmussen [3J Proposition 3.2], Robert [TTJ Proposition 8.12]. 

In the following, W(Q) = — 1. Then (W(i),< > 0) remains in —1 for an exponentially 
distributed duration E\( u ) with parameter A(u), after which it jumps to an a-distributed 
state in R + . Then, its excursion in R + has same distribution as (V(t), t > 0) until it returns 
to — 1. Thus, 7({— 1}) = 1/X(u), and r_i = -Ba(u) + T where r is the first instant of an 
inhomogeneous Poisson process on R + with rate function (fj,(V(t), u),t > 0), and 

P(t > i I V(s), < s < t) = exp ^- ^ /^(s), u) ds 

Since (V(i),t > 0) is Harris ergodic and J fi(x, u)n v (dx) > 0, the corresponding ergodic 
theorem yields 

lim / /i(V(s), u) ds = +00 , a.s., 

so that P(r = 00) = 0, and thus P(r_i = cxd) = implying that 7 is indeed the unique 
invariant measure. From the above, 

f +00 



e(£ f(Y(t))dtj =e(7 °°/(nt))i {T>t} ^ 



+00 



/(V(*))P(t > i I ^(s), <s<t)dt 



E l/(V(*))exp l-y fj,(V(s),u)ds\ J dt 

so that 7 defined by ([S]) is equal to A(w)7. □ 

If the measure 7 has a finite mass then there is a unique stationary distribution given by 
7r := 7/(1 + 7 (!£+)), else there is none. 

Remark 1. The measure 7 is finite on all Borel functions f such that, for some C > 0, 
\f(x)\ < C/j.(x,u) for any x > 0. Indeed, since t — > V(t) is continuous almost everywhere 
for the Lebesgue measure on K + , then 

|/(V(t))|exp(-jT n(y(a),u)ds^ dt 

< c J q v(V(t),u)exp(- J n(V(s),u)dsj dt<C. 
In particular, i/inf x >o fJ,(x,u) > then 7 has a finite mass. 

5.3. Fixed-point equations. In this section, the study of the case of K classes of users is 
resumed, and regular notation is again used. It is assumed that the functions ak(-,u) and 
bk(-,u) satisfy the assumptions of Theorem 15. 1[ and that the process (Wfe(t),l < k < K) 
has a stationary distribution 7r on R , and the corresponding load vector is denoted by 
u = (Uj,l <j< J). 
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Because of the independence of the coordinates, n can be written as n = (££)j, =1 ^k,u where 
7Tk, u is the stationary distribution of the solution (Wk(t),t > 0) of SDE ([2]) with coefficients 
Afc(u) and ru and functions <!/-(•, u), bk(-,u) and /ife(-,w). By Theorem 15. 2[ 

f(x)w k!U (dx) = 1 

1 + \k(u)Z k {u) 



(7) 



Afe(u) 



+oo / / />£ 



i + \k(u)z k (u) y v ' V 



E I /(Vfc iU (t)) exp I - / ^ k (V k , u (s),u)ds) dt, 



with Z k (u) = I E exp - / /Xfc(14, u (s), u) ds dt , 



(i 



for all non-negative Borel functions / on lU { — 1}, where Vk >u is the unique solution of 
the SDE ([3]) (with index k added to the functions) with initial distribution ak(dw), and Z k 
is the appropriate normalization constant. Hence, by rewriting (fTJ) one gets the following 
theorem. 

Theorem 5.3. Let the functions at and bk and Xk and /ik satisfy the assumptions of 
Theorem \5.B for any 1 < k < K and u G Mj ] + . Then, any stationary distribution of the 
nonlinear SDE ([8]) can be written as it = ^k,u, where Hk,u is defined by Equation ([7]) 

and u = (uj , 1 < j < J) € Mi is a solution of the fixed point equation 

(8) Uj = A > kPk i + \ k k (u) Zk ( u ) £ E ( Ffc > llW exp ( _ I ^(V k ,u{^ u ) ds )) dt 

where Vk. u is the solution of the SDE ([3]) associated to rk and the functions ak(-, u), bk{-, u), 
u) with initial distribution a k (dw) and Zk(u) is the corresponding normalizing constant 
given in Equation ([7]). 

The characterization of the invariant distributions involves the transitory behavior of the 
solution (Vk, u (t),t > 0), which leads to a more complex and less explicit expression for the 
fixed-point equation in comparison to the case of permanent connections investigated in 
Graham and Robert [7]. 

One concludes with two examples for the service rates under the assumptions of the 
above theorem. 

Constant service rate. It is assumed that = /i/c(it) for x > and u G MA. Equa- 

tion ([5]) is in this case 

Uj = £ A jkPk fc \ / E [V k , a (t)] e-^ {u)t dt, 1 < j < J. 

k~i l + Xk(u)/Hk(u) Jo 

The integral in the right hand side of the above equation is related to the resolvent of the 
Markov process (Vk, u (t))- 

Linear service rate. It is assumed that fXk(x,u) — x/j,k(u) for x > and u G K+. Equa- 
tion ([8]) becomes 

K 



E, Xk(u)/fik(u 
AjkPk-^ 

k=l 

with 



1 + X k (u)Z k (u) ' 



f + OO 

Z k (u) = I E ( exp ( -fi k (u) I V k ,u(s)ds I ) dt. 
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Future Work. As it may be seen from Relation (jH]), the explicit representation of the 
invariant distribution involves the distribution of the solution of the SDE (J3j) associated to 
a permanent connection. If the equilibrium behavior of this process is fully understood, its 
transient characteristics are not well known. See Chafa'i et al. [4j for the rate of convergence 
to equilibrium. The case of constant rate shows that one should have an expression of the 
resolvent of this process. This is, in our view, an interesting challenging problem of this 
domain. 
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